High dynamic range imaging sensor array

ABSTRACT

An apparatus having a rectangular imaging array characterized by a plurality of pixel sensors and a plurality of readout lines is disclosed. The apparatus has a plurality of column processing circuits, each column processing circuit being connected to a corresponding one of the readout lines and a plurality of signal injectors, one signal injector being connected to each of the readout lines. Each signal injector causes one of a predetermined number of voltages to be coupled to that readout line. An exposure for each of the pixel sensors is determined during image recording periods. The signal injectors inject a plurality of calibration voltages into the readout lines during calibration periods, and determines a gain function of an amplifier in one of the column processing circuits by measuring an output of the amplifier for the plurality of calibration voltages, the calibration period is between the imaging recording periods.

BACKGROUND

CMOS cameras typically form an image of a scene by imaging light fromthe scene onto an array of pixel sensors. Typically, each pixel sensorhas one or more photodetectors. Each photodetector converts the lightreceived during an exposure period into an electrical signal. Theelectrical signal is then digitized by an analog-to-digital converter(ADC) to generate a digital value representing the amount of lightreceived during the exposure time period. The array is typically atwo-dimensional array having thousands of columns and rows of pixels.The array is readout one row at a time using a separate column amplifierand ADC for each row.

In general, a photodiode has some range in which the signal from thephotodiode is a monotonically increasing function of the light receivedin the exposure period. At the bottom of this range, the accuracy of thedetermination of the light intensity from the generated signal islimited by various sources of noise. Above this range, the output of thephotodiode saturates, and hence, intensities above this range cannot beaccurately measured. With current photodiodes, the range of usablesignals is less than that needed to measure all of the intensities inmany images. If the exposure is set to detect low level light signals,the bright regions of the image will be outside the range, and hence,saturated.

Prior art solutions for extending the high range of a pixel sensortypically utilize a second exposure or a second photodiode. In suchschemes, the first exposure or photodiode is set to detect low lightpixels. Pixel sensors that are subjected to high light intensitiessaturate, and hence, cannot provide useful information about the lightintensity in those high light intensity regions of the image. A secondmeasurement is made in a manner that captures the high light intensityregions at the expense of the low light intensity regions. The secondmeasurement can be a second photodiode in the pixel sensor that has amuch lower light sensitivity or a second, shorter exposure, using thesame photodiode. The latter solution is not preferred in motion picturesystems, as the time difference in the two exposures can lead to motionartifacts. The two-photodiode solution has the disadvantage of requiringlarger pixel sensors. However, recent developments in photodiodes haveprovided a second low sensitivity photodiode within a conventionalphotodiode without significantly increasing the size of the pixelsensor.

While providing a second photodiode in each pixel sensor extends thehigh intensity response of a pixel sensor, various noise sources limitthe extent to which the low light regions can be imaged at reasonableexposure times. While shot noise represents the minimum noise floor thatcan be obtained, other sources of noise are still significant, andhence, need to be reduced to further increase the dynamic range of animage sensor.

SUMMARY

The present invention includes an apparatus having a rectangular imagingarray characterized by a plurality of pixel sensors and a plurality ofreadout lines. The apparatus has a plurality of column processingcircuits, each column processing circuit being connected to acorresponding one of the readout lines and a plurality of signalinjectors, one signal injector being connected to each of the readoutlines. Each signal injector causes one of a predetermined number ofvoltages to be coupled to that readout line. A controller determines anexposure for each of the pixel sensors during each of a plurality ofimage recording periods. The controller also causes the signal injectorsto inject a plurality of calibration voltages into the readout linesduring each of a plurality of calibration periods, and determines a gainfunction of an amplifier in one of the column processing circuits bymeasuring an output of the amplifier for the plurality of calibrationvoltages, the calibration periods is between the imaging recordingperiods.

In one aspect of the invention, the controller causes the signalinjectors to inject a signal that has a value that a pixel sensor wouldgenerate if that pixel sensor was not exposed to light, the controllerdetermining a column offset value for each of the column processingcircuits. In another aspect, there are a plurality of rows of signalinjectors, each column processing circuit being connected to a pluralityof the signal injectors. The controller averages the amplifier offsetvalues generated by the signal injectors in determining the columnoffset value. In a still further aspect of the invention, the columnoffset value is determined during the calibration periods.

In a still further aspect of the invention, each of the pixel sensorsincludes first and second photodiodes, the first photodiode ischaracterized by a different light conversion efficiency than the secondphotodiode. In one exemplary embodiment, the second photodiode has alight conversion efficiency less than 1/30th of the first photodiode. Inanother aspect, the second photodiode includes a parasitic photodiodethat includes a floating diffusion node that is also used to convert acharge generated by the first photodiode to a voltage.

In another aspect, the controller determines a ratio of the firstphotodiode light conversion efficiency to the second photodiode lightconversion efficiency during the image recording periods. In one aspectof the invention, the controller determines the ratio by averagingsignals from a plurality of pixel sensors in which the second photodiodegenerates a signal in a calibration range. In another aspect of theinvention, the calibration range excludes pixel sensors in which thesecond photodiode has a dark current greater than a dark currentthreshold.

In another aspect of the invention, the pixel sensors are divided intocolor channels, each color channel having a corresponding color filterover the pixel sensors in that color channel, and the controllerdetermines the ratio separately for each color channel.

In a still further aspect of the invention, the first photodiodemeasures exposures between a first exposure and a second exposure andwherein the second photodiode can measure light exposure between a thirdexposure and a fourth exposure, the third exposure is less than thesecond exposure and the fourth exposure is greater than the secondexposure.

In another aspect of the invention, the controller uses the firstphotodiode to measure exposures less than the second exposure and thesecond photodiode to measure exposures greater than the second exposureto simulate a single photodiode that can measure exposures between thefirst and fourth exposures. In another aspect of the invention, thesimulated single photodiode produces a first exposure value that is alinear function of the exposure and independent of the light conversionefficiencies of the first and second photodiodes and variations in thecolumn processing circuitry for each of the pixel sensors. In a stillfurther aspect of the invention, the first exposure value ischaracterized by a shot noise value and the controller outputs a secondexposure value for each of the pixel sensors, the second exposure valueis determined by the first exposure value, the second exposure valuerequiring fewer bits to output and differing from the first exposurevalue by an amount that is less than the shot noise value.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a two-dimensional imaging array according to oneembodiment of the present invention.

FIG. 2 illustrates a prior art pixel sensor.

FIG. 3 illustrates a pixel sensor in which the parasitic photodiode isutilized in an image measurement.

FIG. 4 illustrates a column amplifier and ADC according to oneembodiment of the present invention.

FIG. 5 illustrates a signal injector that is readout on a readout line83 in response to a row select signal on line 197.

DETAILED DESCRIPTION

To simplify the following discussion, a pixel sensor is defined to be acircuit that converts light incident thereon to an electrical signalhaving a magnitude that is determined by the amount of light that wasincident on that circuit in a period of time, referred to as theexposure. The pixel sensor has a gate that couples that electricalsignal to a readout line in response to a signal on a row select line.

A rectangular imaging array is defined to be a plurality of pixelsensors organized as a plurality of rows and columns of pixel sensors.The rectangular array includes a plurality of readout lines and aplurality of row select lines, each pixel sensor being connected to onerow select line and one readout line, the electrical signal generated bythat pixel being connected to the readout line associated with thatpixel in response to a signal on the row select line associated withthat pixel sensor.

The manner in which the present invention provides its advantages can bemore easily understood with reference to FIG. 1, which illustrates atwo-dimensional imaging array according to one embodiment of the presentinvention. Rectangular imaging array 80 includes a pixel sensor 81. Eachpixel sensor has a main photodiode 86 and a parasitic photodiode 91. Themanner in which the pixel sensor operates will be discussed in moredetail below. The reset circuitry and amplification circuitry in eachpixel is shown at 87. The pixel sensors are arranged as a plurality ofrows and columns. Exemplary rows are shown at 94 and 95. Each pixelsensor in a column is connected to a readout line 83 that is shared byall of the pixel sensors in that column. Each pixel sensor in a row isconnected to a row select line 82 which determines if the pixel sensorin that row is connected to the corresponding readout line.

The operation of rectangular imaging array 80 is controlled by acontroller 92 that receives a pixel address to be readout. Controller 92generates a row select address that is used by row decoder 85 to enablethe readout of the pixel sensors on a corresponding row in rectangularimaging array 80. The column amplifiers are included in an array ofcolumn amplifiers 84 which execute the readout algorithm, which will bediscussed in more detail below. All of the pixel sensors in a given roware readout in parallel; hence there is one column amplification and ADCcircuit per readout line 83. The column processing circuitry will bediscussed in more detail below.

When rectangular imaging array 80 is reset and then exposed to lightduring an imaging exposure, each photodiode accumulates a charge thatdepends on the light exposure and the light conversion efficiency ofthat photodiode. That charge is converted to a voltage by reset andamplification circuitry 87 in that pixel sensor when the row in whichthe pixel sensor associated with that photodiode is readout. Thatvoltage is coupled to the corresponding readout line 83 and processed bythe amplification and ADC circuitry associated with the readout line inquestion to generate a digital value that represents the amount of lightthat was incident on the pixel sensor during the imaging exposure.

Ideally, each pixel sensor is identical to every other pixel sensor, isreset to the same voltage during readout, and generates a signal valueof zero when no light impinges on rectangular imaging array 80. Inaddition, under ideal conditions each column application circuit isidentical to every other column amplification circuit. There are fouranalog conversion factors in the chain of processing from light exposureof a photodiode to a final digital value. These are the light-to-chargeconversion efficiencies of the photodiodes. The charge-to-voltageconversion is in the pixel reset and amplification circuitry 87, andthere is the voltage amplification circuitry in the column processingcircuitry. Differences in these analog conversion factors give rise tofixed pattern noise (FPN). The FPN can depend on factors that changeover time and also depend on the temperature of the imaging array whenthe exposure is taken.

In addition to FPN, there are other noise factors that must be reducedto obtain a noise factor that is small compared to the shot noise. Resetnoise is an example of this type of noise. The manner in which resetnoise is created can be more easily understood with reference to FIG. 2,which illustrates a prior art pixel sensor. FIG. 2 is a schematicdrawing of a typical prior art pixel sensor in one column of pixelsensors in an imaging array. Pixel sensor 21 includes a photodiode 22that measures the light intensity at a corresponding pixel in the image.Initially, photodiode 22 is reset by placing gate 25 in a conductingstate and connecting floating diffusion node 23 to a reset voltage, Vr.Gate 25 is then closed and photodiode 22 is allowed to accumulatephotoelectrons. A potential on gate 27 sets the maximum amount of chargethat can be accumulated on photodiode 22. If more charge is accumulatedthan allowed by the potential on gate 27, the excess charge is shuntedto ground through gate 27.

After photodiode 22 has been exposed, the charge accumulated inphotodiode 22 is typically measured by noting the change in voltage onfloating diffusion node 23 when the accumulated charge from photodiode22 is transferred to floating diffusion node 23. Floating diffusion node23 is characterized by a capacitance represented by capacitor 23′. Inpractice, capacitor 23′ is charged to a voltage Vr and isolated bypulsing the reset line of gate 24 prior to floating diffusion node 23being connected to photodiode 22. The charge accumulated on photodiode22 is transferred to floating diffusion node 23 when gate 25 is opened.The voltage on floating diffusion node 23 is sufficient to remove all ofthis charge, leaving the voltage on floating diffusion node 23 reducedby an amount that depends on the amount of charge transferred and thecapacitance of capacitor 23′. Hence, by measuring the change in voltageon floating diffusion node 23 after gate 25 is opened, the accumulatedcharge can be determined.

If the reset voltage on floating diffusion node 23 is sufficientlyreproducible, then a single measurement of the voltage on floatingdiffusion node after reset is sufficient. However, noise results insmall variations in the reset voltage. If this noise is significant, acorrelated double sampling algorithm is utilized. In this algorithm,floating diffusion node 23 is first reset to Vr using reset gate 24. Thepotential on floating diffusion node 23 is then measured by connectingsource follower 26 to readout line 31 by applying a select signal toline 28 to a readout gate. This reset potential is stored in columnamplifier 32. Next, gate 25 is placed in a conducting state and thecharge accumulated in photodiode 22 is transferred to floating diffusionnode 23. It should be noted that floating diffusion node 23 iseffectively a capacitor that has been charged to Vr. Hence, the chargeleaving photodiode 22 lowers the voltage on floating diffusion node 23by an amount that depends on the capacitance of floating diffusion node23 and the amount of charge that is transferred. The voltage on floatingdiffusion node 23 is again measured after the transfer. The differencein voltage is then used to compute the amount of charge that accumulatedduring the exposure.

The present invention is based on the observation that a pixel of thetype discussed above can be modified to include a second parasiticphotodiode that is part of the floating diffusion node and has asignificant photodiode detection efficiency. This second light detectordoes not significantly increase the size of the pixel, and hence, thepresent invention provides the advantages of a two-photodiode pixelwithout significantly increasing the pixel size.

To distinguish the parasitic photodiode from photodiode 22, photodiode22 and photodiodes serving analogous functions will be referred to asthe “conventional photodiode”. Refer now to FIG. 3, which illustrates apixel sensor in which the parasitic photodiode is utilized in an imagemeasurement. To simplify the following discussion, those elements ofpixel sensor 41 that serve functions analogous to those discussed abovewith respect to FIG. 1 have been given the same numeric designations andwill not be discussed further unless such discussion is necessary toillustrate a new manner in which those elements are utilized. Ingeneral, parasitic photodiode 42 has a detection efficiency that issignificantly less than that of photodiode 22. The manner in which theratio of the photodiode detection efficiencies of the two photodiodes isadjusted is discussed in more detail in co-pending U.S. patentapplication Ser. No. 14/591,873, filed on Jan. 7, 2015. In one exemplaryembodiment, the ratio of the conversion efficiency of the mainphotodiode to the parasitic photodiode is 30:1. Other embodiments inwhich this ratio is 20:1 or 15:1 are useful.

The manner in which pixel sensor 41 is utilized to measure the intensityof a pixel in one embodiment of the present invention will now beexplained in more detail. The process may be more easily understoodstarting from the resetting of the pixel after the last image readoutoperation has been completed. Initially, main photodiode 22 is reset toVr and gate 25 is closed. This also leaves floating diffusion node 43reset to Vr. If a correlated double sampling measurement is to be made,this voltage is measured at the start of the exposure by connectingfloating diffusion node 43 to column amplifier 170. Otherwise, aprevious voltage measurement for the reset voltage is used. During theimage exposure, parasitic photodiode 42 generates photoelectrons thatare stored on floating diffusion node 43. These photoelectrons lower thepotential on floating diffusion node 43. At the end of the exposure, thevoltage on floating diffusion node 43 is measured by connecting theoutput of source follower 26 to column amplifier 170, and the amount ofcharge generated by parasitic photodiode 42 is determined to provide afirst pixel intensity value. Next, floating diffusion node 43 is againreset to Vr and the potential on floating diffusion node 43 is measuredby connecting the output of source follower 26 to column amplifier 170.Gate 25 is then placed in the conducting state and the photoelectronsaccumulated by main photodiode 22 are transferred to floating diffusionnode 43. The voltage on floating diffusion node 43 is then measuredagain and used by column amplifier 170 to compute a second pixelintensity value.

If the light intensity on the corresponding pixel was high, mainphotodiode 22 will have overflowed; however, parasitic photodiode 42,which has a much lower conversion efficiency, will have a value that iswithin the desired range. On the other hand, if the light intensity waslow, there will be insufficient photoelectrons accumulated on parasiticphotodiode 42 to provide a reliable estimate, and the measurement frommain photodiode 22 will be utilized.

The double correlated sampling corrects for reset noise. In addition toreset noise, noise arises from the conversion of the analog voltage onreadout lines 83 to a digital value by the ADC associated with thatreadout line. In the simplest case, the ADC converts the voltage inputthereto to a digital value that is related to the voltage V, by V=NS,where N is the digital value and S is the step size of the ADC. Given anN value and the known value of S, the reconstructed voltage value willdiffer from the original by an error that is half the step size. Thiserror gives rise to noise that will be referred to as digitization noisein the following discussion. This digitization noise is added to theshot noise in the final digital representation of the light exposure foreach pixel. The shot noise is approximately equal to the square root ofthe number of photons that were converted to photoelectrons in thephotodiode. Hence, the shot noise increases with increasing lightexposure. In low light conditions, the shot noise, in absolute terms, issmall, and hence, the digitization noise can be significant if S issmall. However, if S is small, the number of bits that must be used inthe ADC to represent the entire range of input voltages becomes large.Given the large number of ADCs in an imaging array, the increase in costbecomes significant.

In principle, an ADC that has a variable step size can be utilized todigitize the column voltages. However, the additional circuitry forchanging the step size as a function of input voltage increases the costof the ADC. In such an arrangement, the output of the ADC is anon-linear function of the input voltage, small input voltages beingdigitized with a smaller step size. While this arrangement allows thesystem to maintain the digitization noise at a level that is smallcompared to the shot noise, the ADC needs to be able to function overthe entire range of voltage values that may be generated in any image.

The present invention avoids these problems by using a dual gainamplifier to amplify the signal on the corresponding readout line 83. Asingle ADC then digitizes the output of the amplifier. Changing theamplification factor is equivalent to changing the step size of the ADC.In addition, the range of voltages over which the ADC must operate isreduced. Refer now to FIG. 4, which illustrates a column amplifier andADC according to one embodiment of the present invention. This columnprocessing circuit is described in detail in co-pending U.S. patentapplication Ser. No. 14/097,162, filed on Dec. 4, 2013, which is herebyincorporated by reference. For the purposes of the present discussioncolumn processing circuit 70 amplifies and processes the signals on bitline 37. Capacitive transimpedance amplifier 50 is constructed from anoperational amplifier 51 and two feedback capacitors shown at 52 and 53having capacitances C₅₂ and C₅₃, respectively. When switch 54 is open,the gain of capacitive transimpedance amplifier 50 is proportional toC₅₆/C₅₂, where C₅₆ is the capacitance of capacitor 56. When switch 54 isclosed, capacitors 52 and 53 are connected in parallel, and the gain ofcapacitive transimpedance amplifier 50 is proportional to C₅₆/(C₅₂+C₅₃).The state of switch 54 is set by latching comparator 68 that comparesthe output of capacitive transimpedance amplifier 50 with a referencevoltage, V₂. In one embodiment, C₅₆/(C₅₂+C₅₃) is approximately 1, andC₅₆/C₅₂ is between 20 and 30.

In operation, switch 54 is controlled by the output of a latchingcomparator shown at 68 and by controller 92 shown in FIG. 1. Prior toeach voltage measurement on bit line 37, latching comparator 68 is resetand switch 55 is closed to short the input and output of operationalamplifier 51. Initially, switch 54 is open, and operational amplifier 51has its maximum gain. When a signal is transferred to capacitor 56 formeasurement, the output of operational amplifier 51 rises. If the outputof operational amplifier 51 exceeds V₂, latching comparator 68 is setthereby generating a signal on line 67 which is used to close switch 54.The gain of capacitive transimpedance amplifier 50 is thus reduced tothe low value. After capacitive transimpedance amplifier 50 has settled,the output voltage is stored on either capacitor 63 or capacitor 64 indouble sampling circuit 60 depending on the state of switches 61 and 62,respectively. When both the reset value and the value representing thestored charge on the photodiode in the pixel currently connected to bitline 37 are stored on capacitors 64 and 63, respectively, the differencein potential is digitized by ADC 65 and the value output on line 66together with a value indicative of the gain value of capacitivetransimpedance amplifier 50 on line 67.

When the light level stored in the pixel that is connected to bit line37 is low, capacitive transimpedance amplifier 50 and the associatedcorrelated double sampling circuitry behave as a conventional columnprocessing circuit in that the gain of capacitive transimpedanceamplifier 50 is at the high value for both the reset and measurementphases of the correlated double sampling. When the light level is high;however, the gain used to measure the reset potential will be differentthan the gain used to measure the charge that was transferred from thephotodiode. Hence, the difference computation will be in error. In manycases, this does not cause a significant problem, because the correlateddouble sampling computation only provides a significant difference fromthe value that would be obtained by just measuring the charge that wasstored in the photodiode in cases where the photodiode charge is small.However, if a correction for this error is needed, a modified doublesampling circuit in which the observed reset value is divided by anappropriate factor that depends on the difference in gain of the twophases can be utilized.

Capacitive transimpedance amplifier 50 can be viewed as a capacitivetransimpedance amplifier with a variable capacitive feedback circuit asthe feedback loop. The feedback capacitance is set to maintain theoutput signal below a predetermined signal level. While the embodimentshown in FIG. 4 has two gain levels, additional gain levels can be setby providing more feedback capacitors, each with a separately activatedswitch. As will be explained in more detail below, in one embodiment ofthe present invention, capacitive transimpedance amplifier 50 has fourgain levels. Two gains are used for processing the signal from theparasitic photodiode and two gains are used for processing the signalfrom the main photodiode.

As noted above, an ideal pixel sensor would generate a signal of zerowhen no light is directed onto the imaging array. However, in practice,even a dark pixel signal has some small signal. This dark signal canvary from exposure to exposure in response to temperature changes andother factors. In addition, the readout circuitry comprising theamplifiers, correlated double sampling, and ADC in each row can have anon-zero offset. In principle, this source of noise can be reduced byincluding one or more optically black rows in the imaging array bymasking the pixel sensors in those rows. An exemplary optically blackrow is shown at 94 in FIG. 1. In this type of correction scheme, thesignal from this row, or an average of signals from a plurality of suchrows is subtracted from the signals generated by the other non-blackpixel sensors when processing each pixel sensor in a row.

Unfortunately, adequately masking the pixel sensors to provide anoptically black row poses significant challenges as light can bereflected from other portions of the imaging array into the pixelsensors in the optically black row. While this source of noise can beacceptable in a conventional imaging array, it poses significantproblems in an imaging array having the dynamic range of the imagingarrays according to the present invention.

The present invention provides a second “black” signal that can be usedto correct for the offsets in the column processing circuitry. Thissignal is generated by the column calibration circuits 96 shown inFIG. 1. In one aspect of the invention, the calibration circuits includea number of rows of signal injectors. Refer to FIG. 5, which illustratesa signal injector that is readout on a readout line 83 in response to arow select signal on line 197. Signal injector 196 includes a sourcefollower 191 and a select gate 192, that are the same as thecorresponding elements in the pixel sensors. Signal injector 196receives a test signal on bus 193 that is coupled to the gate of sourcefollower 191. Hence, the output of signal injector 196 is a voltage thatreflects the voltage that would be generated by a pixel sensor that hada voltage V_(test) at its gate.

The resulting signal on readout line 83 is processed by thecorresponding column processing circuitry in the same manner as a signalfrom a pixel sensor. In particular, correlated double sampling isapplied during the processing of the signal. That is, V_(test) is firstset to the reset voltage Vr and the signal processed. Next, V_(test) isset to another voltage to provide a test signal that is processed aftersubtracting the previous signal. If the signal is set to Vr during bothsteps, the resulting signal at the ADC in the column processingcircuitry should be zero, which would be the result if a pixel did notreceive any light. Hence, this value will be referred to as electricalblack (EB).

In one aspect of the invention, there are several such signal injectorsconnected to each readout line. The resulting EB signals are averaged toprovide an average EB signal that is subtracted from signals from normalpixel sensors to produce the final pixel sensor values which reflect theactual light exposure received by each pixel sensor. The average EBvalue has reduced noise. The EB values can change slowly withenvironmental variables such as temperature. Hence, a running average ofthe EB values is maintained for each column of pixel sensors. Atpredetermined intervals, additional EB values are measured and added tothis running average and older EB values are discarded.

The signal injectors are also used to calibrate the column processingcircuitry during the operation of the imaging sensor. As noted above,the variations in the amplifiers, ADCs and the other components in thereadout processing circuitry across columns cause Column Fixed PatternNoise (CFPN) in CMOS sensors. CFPN is a major contributor of imagequality degradation in low light and/or low contrast (e.g. anilluminated white paper) scenes. CFPN can be viewed as having twocomponents, offset CFPN and gain CFPN. A column amplifier amplifies theinput signal and adds some offset to that amplified signal. The presentinvention is based on the observation that the offset CFPN isindependent of the input signal to the column amplifier; however, thegain CFPN depends on the input signal magnitude as well as theindividual amplifier, as the gain is not completely constant over therange of voltages presented on the corresponding readout line. The gainfunction of an amplifier is defined to be the gain of the amplifier as afunction of the input voltage to that amplifier. In addition, the gainand offset CFPNs vary with time and environmental variables such as thetemperature of the imaging array.

The subtraction of the average EB signal for a column corrects foroffset CFPN in that column. This is part of the processing of eachframe, and hence, takes into account both fixed offset CFPN and changesin the fixed offset CFPN over time and other slowly varyingenvironmental factors.

In prior art imaging arrays, the gain CFPN is corrected usingpre-calibrated coefficient(s) for each column after an offset CFPNcorrection has been made. The calibration step is usually fulfilled atthe factory and stays unchanged for the camera life. Hence, thisapproach does not correct for the temporal gain variation. As a result,significant gain CFPN is still present.

The present invention includes a dynamic gain CFPN compensation schemein addition to the offset CFPN correction. The amplifiers in the columnprocessing circuitry have four nominal gain settings. Two of these gainsettings, referred to as the high and low main photodiode gain settings,are used to process the signal from the main photodiode. Similarly, twoof these gain settings, referred to as the high and low parasiticphotodiode gain settings, are used to process the signal from theparasitic photodiode. Hence, there are four amplifier gains that must becalibrated and gain as a function of voltage stored for each amplifiergain.

Referring again to FIG. 1, rectangular imaging array 80 includes columncalibration circuits 96 that generate calibration signals which are fedinto column amplifiers and the downstream circuits when the amplifiersare set to each of the gain settings. In one aspect of the presentinvention, the injectors discussed above are used to generate knownvoltages on the readout lines to provide the calibration signals. Thecalibration signals are also processed using double correlated sampling;however, the second voltage in the sequence is set to a voltage below Vrto provide a signal of known magnitude, so that the processed value ofthe signal through the column processing circuitry can be determined.The resulting offset and gain profiles are stored in a memory that ispart of the system controller. The different calibration signal levelsare generated in the background when the sensor is running. Then acorrection algorithm is applied by using these stored profiles tocorrect the CFPN. Since these profiles are dynamically generated and thecorrection algorithm keeps running in the background as the sensor isrunning, the system controller is able to track column variations andapply the corresponding compensations when sensor running conditionschange (e.g., temperature, supply voltage, etc.).

As noted above, the present invention utilizes pixel sensors having twophotodiodes per pixel sensor, a main photodiode and a parasiticphotodiode. The main photodiode is adapted for low light detection, andhence, has a high light conversion gain and is a pinned photodiode toreduce noise. The parasitic photodiode is adapted for high lightdetection and has a low light conversion gain. In addition, the signalsfrom each of the photodiodes can be the results obtained with the twodifferent column gain levels. These results are combined to generate adigital light measurement that would have been obtained if the mainphotodiode had the extended range and the signal from that photodiodewas processed using a single amplification gain.

The signal from the low sensitivity parasitic photodiode extends theuseful range of the pixel sensor. When the parasitic photodiode issupplying the light intensity value, the parasitic photodiodemeasurement needs to be converted to a value that would have beenobtained from the main photodiode if the main photodiode did notsaturate. To provide this extension, the relative gains of the twophotodiodes need to be known. The ratio of the two gains depends on theaverage wavelength of the light received by the photodiodes, and hence,must be calibrated for the different color channels in the imagingarray. However, even within a given color channel, there are variationsthat depend on the color temperature of the incident light. Hence, inthe present invention, the ratio is calibrated for each image.

The relative sensitivities of the main and parasitic photodiodes are setsuch that there is a range of incident light intensities that provideuseful signals for both photodiodes in the same pixel sensor. To besuitable for the calibration, the light intensity must be between afirst intensity value that is less than the intensity at which the mainphotodiode saturates and a second intensity value that is greater thanthe minimum intensity at which the parasitic photodiode provides ameaningful signal. During the readout of the imaging array, the EBoffset is removed from the column signals and those signals that arewithin the calibration range are identified. The ratio of the twophotodiode signals for these pixels is computed and added to a runningaverage calibration ratio that is used to compute the light intensityfor all the pixel sensors in that color channel for which the parasiticphotodiode signal provides the light measurement.

In one aspect of the invention, each pixel in the final image iscomputed from the output of four pixel sensors that are adjacent to oneanother. Two of the pixel sensors, G1 and G2, are covered by greenfilters and the remaining two pixel sensors, R and B, are covered by redand blue filters, respectively. The pixel sensors that are covered bythe same color pixel within the imaging array are referred to as a“color channel”. The pixel sensors have some degree of cross-talk. Thatis, a light input in the red spectral region generates a non-zeroresponse in the other color pixel sensors in the four pixel group. Ithas been observed that the cross-talk is different for G1 and G2. Hence,the calibration ratio is computed separately for each of the greensensors; that is, the two green sensors are treated as separate colorchannels in this aspect of the present invention.

The calibration ratio of the main photodiode to the parasitic photodiodedepends on the dark current in both photodiodes being negligible. Whilethis assumption is true for the main photodiode, the dark current in theparasitic photodiodes can vary beyond the tolerable limit. Pixel sensorsin which the parasitic photodiode has a large dark current will bereferred to as “hot pixels”. These pixels must be excluded from therunning average for the calibration ratio in each color channel. In oneaspect of the invention, calibration ratio values that are outliers inthe statistical distribution of calibration ratio values are not used tocompute the running average.

If a particular embodiment of an imaging array according to the presentinvention has sufficient memory, the controller can store a list of hotpixels as determined by a calibration procedure that is performed at thefactory. In this case, calibration ratios for these pixels are neverused to provide the running average.

For each pixel sensor, the parasitic and main photodiode signals aredigitized. The digital value includes the amplification gain used by thecolumn amplifier prior to digitization. Assume that the gain of theamplifier was constant over the corresponding input voltage ranges. Thenthe light intensity is the product of a “step size” related to the gainand the digital value from the ADC. As noted above, the gain is notnecessarily constant for a given amplifier gain, but depends to someextent on the input voltage, and hence, the digital value. Controller 92stores a table of gains for each column amplifier.

As noted above, the goal of the blending of the outputs from the mainphotodiode and the parasitic photodiode is to provide a signal valuethat is a linear function of the light exposure at the correspondingpixel sensor. The dynamic range of an image sensor according to thepresent invention can be as high as 10⁶. To adequately represent thislinear value over the entire range of exposures a 24-bit integer isrequired. Hence, the output bus from controller 92 would need to be a24-bit bus. The power needed to drive such a large bus at the speedsneeded in a surveillance or other motion picture camera is significant.Hence, in one aspect of the invention, the final exposure value iscompressed to a much smaller number of bits.

At low exposures, the most significant bits of the digital value arezeros. At high exposures, the digital values in the high bits areimportant; however, the values in the least significant bits aredominated by shot noise, and hence, provide little useful information.Hence, those values can be replaced by zeros or any other value withoutsignificantly altering the exposure values. In one aspect of theinvention, the linear exposure value is transformed to a compressedexposure value using a non-linear transformation that is chosen suchthat the highest compressed digital value requires significantly fewerbits than the non-compressed digital value.

Consider a table of threshold V_(i) values. If an exposure digital, V,is greater than or equal to V_(i) and less than V_(i+1), V is replacedby i. Here, i=1 to Nt. Upon decompression, V is replaced by V_(i). Thetable values, V_(i) are chosen such that the difference between V_(i)and V_(i+1) is less than the shot noise in a signal having the valueV_(i). In addition, the number of entries in the table are chosen suchthat Nt<<V_(max), where V_(max) is the largest pixel signal. In anexemplary embodiment, the linear exposure values require 24 bits, butthe maximum value of i requires only 14 bits. Hence, a substantialsaving in the number of output values is achieved.

The above-described embodiments utilize pixel sensors having a mainphotodiode and parasitic photodiode. However, the teachings of thepresent invention can be applied to reduce the CFPN in imaging arrays inwhich the pixel sensors have two conventional photodiodes. In general,the first photodiode can measure exposures in a first band of exposurescharacterized by a first and a second exposure limit. The secondphotodiode measures exposures in a second band of exposurescharacterized by a third and a fourth exposure limit. The first andsecond bands overlap; that is, the third exposure limit is greater thanthe first exposure limit and less than the second exposure limit, andthe fourth exposure limit is greater than the second exposure limit. Thepresent invention preferably uses the parasitic photodiode as the secondphotodiode because the resulting imaging sensor is substantiallysmaller, and hence, less expensive than an imaging sensor that utilizestwo conventional photodiodes and an extra internal gate to determinewhich photodiode is currently connected to the floating diffusion node.

The above-described embodiments of the present invention have beenprovided to illustrate various aspects of the invention. However, it isto be understood that different aspects of the present invention thatare shown in different specific embodiments can be combined to provideother embodiments of the present invention. In addition, variousmodifications to the present invention will become apparent from theforegoing description and accompanying drawings. Accordingly, thepresent invention is to be limited solely by the scope of the followingclaim.

What is claimed is:
 1. An apparatus comprising a rectangular imagingarray characterized by a plurality of pixel sensors and a plurality ofreadout lines; a plurality of column processing circuits, each columnprocessing circuit being connected to a corresponding one of saidplurality of readout lines; a plurality of signal injectors, one signalinjector being connected to each of said readout lines, each signalinjector causing one of a predetermined number of voltages to be coupledto that readout line; and a controller that determines an exposure foreach of said plurality of pixel sensors during each of a plurality ofimage recording periods, causes said signal injectors to inject aplurality of calibration voltages into said readout lines during each ofa plurality of calibration periods, and determines a gain function of anamplifier in one of said plurality of column processing circuits bymeasuring an output of said amplifier during said plurality ofcalibration periods, said plurality of calibration periods being betweensaid image recording periods.
 2. The apparatus of claim 1 wherein saidcontroller causes said signal injectors to inject a signal that has avalue that a pixel sensor would generate if that pixel sensor was notexposed to light, said controller determining a column offset value foreach of said plurality of column processing circuits.
 3. The apparatusof claim 2 further comprising a plurality of rows of signal injectors,each column processing circuit being connected to a plurality of saidsignal injectors, said controller averaging said column offset valuesgenerated by said signal injectors in determining said column offsetvalue.
 4. The apparatus of claim 2 wherein said column offset value isdetermined during plurality of said calibration periods.
 5. Theapparatus of claim 1 wherein each of said plurality of pixel sensorscomprises first and second photodiodes, said first photodiode beingcharacterized by a different light conversion efficiency than saidsecond photodiode.
 6. The apparatus of claim 5 wherein said secondphotodiode has a light conversion efficiency less than 1/30th of saidfirst photodiode.
 7. The apparatus of claim 6 wherein said secondphotodiode comprises a parasitic photodiode that includes a floatingdiffusion node that is also used to convert a charge generated by saidfirst photodiode to a voltage.
 8. The apparatus of claim 5 wherein saidcontroller determines a ratio of said first photodiode light conversionefficiency to said second photodiode light conversion efficiency duringsaid plurality of image recording periods.
 9. The apparatus of claim 8wherein said controller determines said ratio by averaging signals froma plurality of pixel sensors in which said second photodiode generates asignal in a calibration range.
 10. The apparatus of claim 9 wherein saidcalibration range excludes pixel sensors in which said second photodiodehas a dark current greater than a dark current threshold.
 11. Theapparatus of claim 8 wherein said plurality of pixel sensors are dividedinto color channels, each color channel having a corresponding colorfilter over pixel sensors in that color channel, and wherein saidcontroller determines said ratio separately for each of said colorchannels.
 12. The apparatus of claim 5 wherein said first photodiodemeasures exposures between a first exposure and a second exposure andwherein said second photodiode can measure light exposure between athird exposure and a fourth exposure, said third exposure being lessthan said second exposure and said fourth exposure being greater thansaid second exposure.
 13. The apparatus of claim 5 wherein saidcontroller uses said first photodiode to measure exposures less thansaid second exposure and said second photodiode to measure exposuresgreater than said second exposure to simulate a single photodiode thatcan measure exposures between said first and fourth exposures.
 14. Theapparatus of claim 13 wherein said simulated single photodiode producesa first exposure value that is a linear function of said exposure andindependent of said light conversion efficiencies of said first andsecond photodiodes and variations in said plurality of column processingcircuits.
 15. The apparatus of claim 14 wherein said first exposurevalue is characterized by a shot noise value and said controller outputsa second exposure value for each of said plurality of pixel sensors,said second exposure being determined by said first exposure value, saidsecond exposure requiring fewer bits to output and differing from saidfirst exposure by an amount that is less than said shot noise value.